{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DjUA6S30k52h"
      },
      "source": [
        "##### Copyright 2022 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SpNWyqewk8fE"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6x1ypzczQCwy"
      },
      "source": [
        "# Using TensorFlow Recommenders with TFX\n",
        "\n",
        "***A tutorial to train a TensorFlow Recommenders ranking model as a [TFX pipeline](https://www.tensorflow.org/tfx).***"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HU9YYythm0dx"
      },
      "source": [
        "\u003ctable class=\"tfo-notebook-buttons\" align=\"left\"\u003e\n",
        "  \u003ctd\u003e\n",
        "    \u003ca target=\"_blank\" href=\"https://www.tensorflow.org/recommenders/examples/ranking_tfx\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" /\u003eView on TensorFlow.org\u003c/a\u003e\n",
        "  \u003c/td\u003e\n",
        "  \u003ctd\u003e\n",
        "    \u003ca target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/recommenders/blob/main/docs/examples/ranking_tfx.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" /\u003eRun in Google Colab\u003c/a\u003e\n",
        "  \u003c/td\u003e\n",
        "  \u003ctd\u003e\n",
        "    \u003ca target=\"_blank\" href=\"https://github.com/tensorflow/recommenders/blob/main/docs/examples/ranking_tfx.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" /\u003eView source on GitHub\u003c/a\u003e\n",
        "  \u003c/td\u003e\n",
        "  \u003ctd\u003e\n",
        "    \u003ca href=\"https://storage.googleapis.com/tensorflow_docs/recommenders/docs/examples/ranking_tfx.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/download_logo_32px.png\" /\u003eDownload notebook\u003c/a\u003e\n",
        "  \u003c/td\u003e\n",
        "\u003c/table\u003e"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_VuwrlnvQJ5k"
      },
      "source": [
        "In this notebook-based tutorial, we will create and run a [TFX pipeline](https://www.tensorflow.org/tfx)\n",
        "to train a ranking model to predict movie ratings using TensorFlow Recommenders (TFRS).\n",
        "The pipeline will consist of three essential TFX components: ExampleGen,\n",
        "Trainer and Pusher. The pipeline includes the most minimal ML workflow like\n",
        "importing data, training a model and exporting the trained TFRS ranking model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Fmgi8ZvQkScg"
      },
      "source": [
        "## Set Up\n",
        "We first need to install the TFX Python package and download\n",
        "the dataset which we will use for our model.\n",
        "\n",
        "### Upgrade Pip\n",
        "\n",
        "To avoid upgrading Pip in a system when running locally,\n",
        "check to make sure that we are running in Colab.\n",
        "Local systems can of course be upgraded separately."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "as4OTe2ukSqm"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "if 'google.colab' in sys.modules:\n",
        "  !pip install --upgrade pip"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MZOYTt1RW4TK"
      },
      "source": [
        "### Install TFX\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iyQtljP-qPHY"
      },
      "outputs": [],
      "source": [
        "!pip install -U tfx\n",
        "!pip install -U tensorflow-recommenders"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EwT0nov5QO1M"
      },
      "source": [
        "### Did you restart the runtime?\n",
        "\n",
        "If you are using Google Colab, the first time that you run\n",
        "the cell above, you must restart the runtime by clicking\n",
        "above \"RESTART RUNTIME\" button or using \"Runtime \u003e Restart\n",
        "runtime ...\" menu. This is because of the way that Colab\n",
        "loads packages.\n",
        "\n",
        "Before we define the pipeline, we need to write the model code for the\n",
        "Trainer component and save it in a file."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BDnPgN8UJtzN"
      },
      "source": [
        "Check the TensorFlow and TFX versions."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6jh7vKSRqPHb"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "print('TensorFlow version: {}'.format(tf.__version__))\n",
        "from tfx import v1 as tfx\n",
        "print('TFX version: {}'.format(tfx.__version__))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aDtLdSkvqPHe"
      },
      "source": [
        "### Set up variables\n",
        "\n",
        "There are some variables used to define a pipeline. You can customize these\n",
        "variables as you want. By default all output from the pipeline will be\n",
        "generated under the current directory. Instead of using the SchemaGen component to generate a schema, for this\n",
        "tutorial we will create a hardcoded schema."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EcUseqJaE2XN"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "PIPELINE_NAME = 'TFRS-ranking'\n",
        "\n",
        "# Directory where MovieLens 100K rating data lives\n",
        "DATA_ROOT = os.path.join('data', PIPELINE_NAME)\n",
        "# Output directory to store artifacts generated from the pipeline.\n",
        "PIPELINE_ROOT = os.path.join('pipelines', PIPELINE_NAME)\n",
        "# Path to a SQLite DB file to use as an MLMD storage.\n",
        "METADATA_PATH = os.path.join('metadata', PIPELINE_NAME, 'metadata.db')\n",
        "# Output directory where created models from the pipeline will be exported.\n",
        "SERVING_MODEL_DIR = os.path.join('serving_model', PIPELINE_NAME)\n",
        "\n",
        "from absl import logging\n",
        "logging.set_verbosity(logging.INFO)  # Set default logging level."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8F2SRwRLSYGa"
      },
      "source": [
        "### Prepare example data\n",
        "Since TFX does not currently support TensorFlow Datasets API, we will download the MovieLens 100K dataset manually for use in our TFX pipeline. The dataset we\n",
        "are using is\n",
        "[MovieLens 100K Dataset](https://grouplens.org/datasets/movielens/100k/).\n",
        "\n",
        "There are four numeric features in this dataset:\n",
        "\n",
        "- userId\n",
        "- movieId\n",
        "- rating\n",
        "- timestamp\n",
        "\n",
        "We will build a ranking model which predicts the `rating` of the movies. We will not use the `timestamp` feature."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "11J7XiCq6AFP"
      },
      "source": [
        "Because TFX ExampleGen reads inputs from a directory, we need to create a\n",
        "directory and copy dataset to it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4fxMs6u86acP"
      },
      "outputs": [],
      "source": [
        "!wget https://files.grouplens.org/datasets/movielens/ml-100k.zip\n",
        "!mkdir -p {DATA_ROOT}\n",
        "!unzip ml-100k.zip\n",
        "!echo 'userId,movieId,rating,timestamp' \u003e {DATA_ROOT}/ratings.csv\n",
        "!sed 's/\\t/,/g' ml-100k/u.data \u003e\u003e {DATA_ROOT}/ratings.csv"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ASpoNmxKSQjI"
      },
      "source": [
        "Take a quick look at the CSV file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-eSz28UDSnlG"
      },
      "outputs": [],
      "source": [
        "!head {DATA_ROOT}/ratings.csv"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OTtQNq1DdVvG"
      },
      "source": [
        "You should be able to see four values. For example, the first example means user '196' gives a rating of 3 to movie '242'."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nH6gizcpSwWV"
      },
      "source": [
        "## Create a pipeline\n",
        "\n",
        "TFX pipelines are defined using Python APIs. We will define a pipeline which\n",
        "consists of following three components.\n",
        "- CsvExampleGen: Reads in data files and convert them to TFX internal format\n",
        "for further processing. There are multiple\n",
        "[ExampleGen](https://www.tensorflow.org/tfx/guide/examplegen)s for various\n",
        "formats. In this tutorial, we will use CsvExampleGen which takes CSV file input.\n",
        "- Trainer: Trains an ML model.\n",
        "[Trainer component](https://www.tensorflow.org/tfx/guide/trainer) requires a\n",
        "model definition code from users. You can use TensorFlow APIs to specify how to\n",
        "train a model and save it in a _saved_model_ format.\n",
        "- Pusher: Copies the trained model outside of the TFX pipeline.\n",
        "[Pusher component](https://www.tensorflow.org/tfx/guide/pusher) can be thought\n",
        "of an deployment process of the trained ML model.\n",
        "\n",
        "Before actually define the pipeline, we need to write a model code for the\n",
        "Trainer component first."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lOjDv93eS5xV"
      },
      "source": [
        "### Write model training code\n",
        "\n",
        "We will build a simple ranking model to predict movie ratings. This model training code will be saved to a separate file.\n",
        "\n",
        "In this tutorial we will use\n",
        "[Generic Trainer](https://www.tensorflow.org/tfx/guide/trainer#generic_trainer)\n",
        "of TFX which support Keras-based models. You need to write a Python file\n",
        "containing `run_fn` function, which is the entrypoint for the `Trainer`\n",
        "component."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aES7Hv5QTDK3"
      },
      "outputs": [],
      "source": [
        "_trainer_module_file = 'tfrs_ranking_trainer.py'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HFsQCOytiidq"
      },
      "source": [
        "The ranking model we use is almost exactly the same as in the [Basic Ranking](https://www.tensorflow.org/recommenders/examples/basic_ranking) tutorial. The only difference is that we use movie IDs instead of movie titles in the candidate tower."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Gnc67uQNTDfW"
      },
      "outputs": [],
      "source": [
        "%%writefile {_trainer_module_file}\n",
        "\n",
        "from typing import Dict, Text\n",
        "from typing import List\n",
        "\n",
        "import numpy as np\n",
        "import tensorflow as tf\n",
        "\n",
        "from tensorflow_metadata.proto.v0 import schema_pb2\n",
        "import tensorflow_recommenders as tfrs\n",
        "from tensorflow_transform.tf_metadata import schema_utils\n",
        "from tfx import v1 as tfx\n",
        "from tfx_bsl.public import tfxio\n",
        "\n",
        "_FEATURE_KEYS = ['userId', 'movieId']\n",
        "_LABEL_KEY = 'rating'\n",
        "\n",
        "_FEATURE_SPEC = {\n",
        "    **{\n",
        "        feature: tf.io.FixedLenFeature(shape=[1], dtype=tf.int64)\n",
        "        for feature in _FEATURE_KEYS\n",
        "    }, _LABEL_KEY: tf.io.FixedLenFeature(shape=[1], dtype=tf.int64)\n",
        "}\n",
        "\n",
        "\n",
        "class RankingModel(tf.keras.Model):\n",
        "\n",
        "  def __init__(self):\n",
        "    super().__init__()\n",
        "    embedding_dimension = 32\n",
        "\n",
        "    unique_user_ids = np.array(range(943)).astype(str)\n",
        "    unique_movie_ids = np.array(range(1682)).astype(str)\n",
        "\n",
        "    # Compute embeddings for users.\n",
        "    self.user_embeddings = tf.keras.Sequential([\n",
        "        tf.keras.layers.Input(shape=(1,), name='userId', dtype=tf.int64),\n",
        "        tf.keras.layers.Lambda(lambda x: tf.as_string(x)),\n",
        "        tf.keras.layers.StringLookup(\n",
        "            vocabulary=unique_user_ids, mask_token=None),\n",
        "        tf.keras.layers.Embedding(\n",
        "            len(unique_user_ids) + 1, embedding_dimension)\n",
        "    ])\n",
        "\n",
        "    # Compute embeddings for movies.\n",
        "    self.movie_embeddings = tf.keras.Sequential([\n",
        "        tf.keras.layers.Input(shape=(1,), name='movieId', dtype=tf.int64),\n",
        "        tf.keras.layers.Lambda(lambda x: tf.as_string(x)),\n",
        "        tf.keras.layers.StringLookup(\n",
        "            vocabulary=unique_movie_ids, mask_token=None),\n",
        "        tf.keras.layers.Embedding(\n",
        "            len(unique_movie_ids) + 1, embedding_dimension)\n",
        "    ])\n",
        "\n",
        "    # Compute predictions.\n",
        "    self.ratings = tf.keras.Sequential([\n",
        "        tf.keras.layers.Dense(256, activation='relu'),\n",
        "        tf.keras.layers.Dense(64, activation='relu'),\n",
        "        tf.keras.layers.Dense(1)\n",
        "    ])\n",
        "\n",
        "  def call(self, inputs):\n",
        "\n",
        "    user_id, movie_id = inputs\n",
        "\n",
        "    user_embedding = self.user_embeddings(user_id)\n",
        "    movie_embedding = self.movie_embeddings(movie_id)\n",
        "\n",
        "    return self.ratings(tf.concat([user_embedding, movie_embedding], axis=2))\n",
        "\n",
        "\n",
        "class MovielensModel(tfrs.models.Model):\n",
        "\n",
        "  def __init__(self):\n",
        "    super().__init__()\n",
        "    self.ranking_model: tf.keras.Model = RankingModel()\n",
        "    self.task: tf.keras.layers.Layer = tfrs.tasks.Ranking(\n",
        "        loss=tf.keras.losses.MeanSquaredError(),\n",
        "        metrics=[tf.keras.metrics.RootMeanSquaredError()])\n",
        "\n",
        "  def call(self, features: Dict[str, tf.Tensor]) -\u003e tf.Tensor:\n",
        "    return self.ranking_model((features['userId'], features['movieId']))\n",
        "\n",
        "  def compute_loss(self,\n",
        "                   features: Dict[Text, tf.Tensor],\n",
        "                   training=False) -\u003e tf.Tensor:\n",
        "\n",
        "    labels = features[1]\n",
        "    rating_predictions = self(features[0])\n",
        "\n",
        "    # The task computes the loss and the metrics.\n",
        "    return self.task(labels=labels, predictions=rating_predictions)\n",
        "\n",
        "\n",
        "def _input_fn(file_pattern: List[str],\n",
        "              data_accessor: tfx.components.DataAccessor,\n",
        "              schema: schema_pb2.Schema,\n",
        "              batch_size: int = 256) -\u003e tf.data.Dataset:\n",
        "  return data_accessor.tf_dataset_factory(\n",
        "      file_pattern,\n",
        "      tfxio.TensorFlowDatasetOptions(\n",
        "          batch_size=batch_size, label_key=_LABEL_KEY),\n",
        "      schema=schema).repeat()\n",
        "\n",
        "\n",
        "def _build_keras_model() -\u003e tf.keras.Model:\n",
        "  return MovielensModel()\n",
        "\n",
        "\n",
        "# TFX Trainer will call this function.\n",
        "def run_fn(fn_args: tfx.components.FnArgs):\n",
        "  \"\"\"Train the model based on given args.\n",
        "\n",
        "  Args:\n",
        "    fn_args: Holds args used to train the model as name/value pairs.\n",
        "  \"\"\"\n",
        "  schema = schema_utils.schema_from_feature_spec(_FEATURE_SPEC)\n",
        "\n",
        "  train_dataset = _input_fn(\n",
        "      fn_args.train_files, fn_args.data_accessor, schema, batch_size=8192)\n",
        "  eval_dataset = _input_fn(\n",
        "      fn_args.eval_files, fn_args.data_accessor, schema, batch_size=4096)\n",
        "\n",
        "  model = _build_keras_model()\n",
        "\n",
        "  model.compile(optimizer=tf.keras.optimizers.Adagrad(learning_rate=0.1))\n",
        "\n",
        "  model.fit(\n",
        "      train_dataset,\n",
        "      steps_per_epoch=fn_args.train_steps,\n",
        "      epochs = 3,\n",
        "      validation_data=eval_dataset,\n",
        "      validation_steps=fn_args.eval_steps)\n",
        "\n",
        "  model.save(fn_args.serving_model_dir)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "blaw0rs-emEf"
      },
      "source": [
        "Now you have completed all preparation steps to build the TFX pipeline."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w3OkNz3gTLwM"
      },
      "source": [
        "### Write a pipeline definition\n",
        "\n",
        "We define a function to create a TFX pipeline. A `Pipeline` object\n",
        "represents a TFX pipeline which can be run using one of pipeline\n",
        "orchestration systems that TFX supports.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "M49yYVNBTPd4"
      },
      "outputs": [],
      "source": [
        "def _create_pipeline(pipeline_name: str, pipeline_root: str, data_root: str,\n",
        "                     module_file: str, serving_model_dir: str,\n",
        "                     metadata_path: str) -\u003e tfx.dsl.Pipeline:\n",
        "  \"\"\"Creates a three component pipeline with TFX.\"\"\"\n",
        "  # Brings data into the pipeline.\n",
        "  example_gen = tfx.components.CsvExampleGen(input_base=data_root)\n",
        "\n",
        "  # Uses user-provided Python function that trains a model.\n",
        "  trainer = tfx.components.Trainer(\n",
        "      module_file=module_file,\n",
        "      examples=example_gen.outputs['examples'],\n",
        "      train_args=tfx.proto.TrainArgs(num_steps=12),\n",
        "      eval_args=tfx.proto.EvalArgs(num_steps=24))\n",
        "\n",
        "  # Pushes the model to a filesystem destination.\n",
        "  pusher = tfx.components.Pusher(\n",
        "      model=trainer.outputs['model'],\n",
        "      push_destination=tfx.proto.PushDestination(\n",
        "          filesystem=tfx.proto.PushDestination.Filesystem(\n",
        "              base_directory=serving_model_dir)))\n",
        "\n",
        "  # Following three components will be included in the pipeline.\n",
        "  components = [\n",
        "      example_gen,\n",
        "      trainer,\n",
        "      pusher,\n",
        "  ]\n",
        "\n",
        "  return tfx.dsl.Pipeline(\n",
        "      pipeline_name=pipeline_name,\n",
        "      pipeline_root=pipeline_root,\n",
        "      metadata_connection_config=tfx.orchestration.metadata\n",
        "      .sqlite_metadata_connection_config(metadata_path),\n",
        "      components=components)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mJbq07THU2GV"
      },
      "source": [
        "## Run the pipeline\n",
        "\n",
        "TFX supports multiple orchestrators to run pipelines.\n",
        "In this tutorial we will use `LocalDagRunner` which is included in the TFX\n",
        "Python package and runs pipelines on local environment."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7mp0AkmrPdUb"
      },
      "source": [
        "Now we create a `LocalDagRunner` and pass a `Pipeline` object created from the\n",
        "function we already defined.\n",
        "\n",
        "The pipeline runs directly and you can see logs for the progress of the pipeline including ML model training."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fAtfOZTYWJu-"
      },
      "outputs": [],
      "source": [
        "tfx.orchestration.LocalDagRunner().run(\n",
        "  _create_pipeline(\n",
        "      pipeline_name=PIPELINE_NAME,\n",
        "      pipeline_root=PIPELINE_ROOT,\n",
        "      data_root=DATA_ROOT,\n",
        "      module_file=_trainer_module_file,\n",
        "      serving_model_dir=SERVING_MODEL_DIR,\n",
        "      metadata_path=METADATA_PATH))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ppERq0Mj6xvW"
      },
      "source": [
        "You should see \"INFO:absl:Component Pusher is finished.\" at the end of the\n",
        "logs if the pipeline finished successfully. Because `Pusher` component is the\n",
        "last component of the pipeline.\n",
        "\n",
        "The pusher component pushes the trained model to the `SERVING_MODEL_DIR` which\n",
        "is the `serving_model/TFRS-ranking` directory if you did not change the\n",
        "variables in the previous steps. You can see the result from the file browser\n",
        "in the left-side panel in Colab, or using the following command:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NTHROkqX6yHx"
      },
      "outputs": [],
      "source": [
        "# List files in created model directory.\n",
        "!ls -R {SERVING_MODEL_DIR}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W8HQfT-ziids"
      },
      "source": [
        "Now we can test the ranking model by computing predictions for a user and a movie:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5EDMkz8Wiidt"
      },
      "outputs": [],
      "source": [
        "import glob\n",
        "# Load the latest model for testing\n",
        "loaded = tf.saved_model.load(max(glob.glob(os.path.join(SERVING_MODEL_DIR, '*/')), key=os.path.getmtime))\n",
        "print(loaded({'userId': [[42]], 'movieId': [[15]]}).numpy())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "08R8qvweThRf"
      },
      "source": [
        "This concludes the TensorFlow Recommenders + TFX tutorial."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [
        "DjUA6S30k52h"
      ],
      "name": "ranking_tfx.ipynb",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.11"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
